High resolution and high sensitivity optically activated motion detection device using multiple color light sources

ABSTRACT

An optical computer mouse senses movement by detecting the variations in intensity of reflected primary colors from a surface over which the mouse is moved. The surface is illuminated by a plurality of colors of light. An image sensor is formed from an array of photodiodes covered with a color filter array that matches the primary colors of the lights and is used to detect intensity of reflected light of the primary colors from the surface on which the optical computer mouse rests. Variations in the intensity of at least one of the primary colors of reflected light are used to determine movement by the computer mouse. Both the intensity of the individual lights and sensitivity of the array of photo diodes are controlled by a controller.

This application claims benefit to U.S. Provisional Patent Application61/611,116, filed on Mar. 15, 2012, which is herein incorporated byreference in its entirety.

RELATED PATENT APPLICATION

This application is related to patent application Ser. No. 13/482,086,filed on May 29, 2012, which is herein incorporated by reference in itsentirety.

1. TECHNICAL FIELD

The present disclosure relates to a motion detection device, and inparticular to an optically activated cursor maneuvering device thatmoves on a two dimensional plane, e.g. a mouse pad or desktop surface,and maneuvers the cursors in either two dimensional or three dimensionaldisplaying devices.

2. BACKGROUND

Today various kinds of electronic systems such as industrial computers,work stations, desktop computers, laptop computers, cell phones etc.rely on a decades old device, an optical mouse, to sense the movement ofthe operator's hand (or finger, or something of equivalent function),and translate that movement into instruction(s) or motion vector(s) ofthe cursor(s) in the display of the electronic system. Inside eachoptical mouse lies an image sensor to capture series of images andcompare them at a high frame rate. There are several factors affectingthe performance of the image sensor, e.g. photodiode size (i.e. pixelsize), resolution (e.g. pixels/inch), frame rate, lens quality, theintensity or incident angle of the illuminating light, etc. In theoptical mouse industry, resolution is defined by the number of pixelsthat the image sensor and lens can view per inch while the mouse isbeing moved. If the resolution is high, then the operator would requireless mouse movement to get accurate motion vector data, and this in turnwill relax the requirement for higher frame rate, which eventuallylowers the power consumption of the optical mouse. Today most of themice run at frame rate of thousands per second, and at a resolution of700-800 pixels per inch. Gaming mice have a higher resolution (e.g.thousands of pixels per inch) as compared to that of normal mice for asmoother and more accurate operation. The above specifications aresatisfactory for most household users where the desktop surface is awood plate, but when the optical mouse is facing a very smooth surface(e.g. a glossy plate), it becomes clumsy, slow, and often leads tounstable results. Therefore, a lot of power is wasted due to many ofcalculations, which only provide erroneous results.

FIG. 1 shows a conventional optical mouse 102 having a select button 101and an USB cable 105 to connect to a computer. A monochromatic lightsource 103 is mounted in the cavity 106. By reflection, light from lightsource 103 impinging onto the targeted desktop surface 107 is capturedby the image sensor 104.

FIG. 2 shows an exemplary case of how a particle on a surface under anoptical computer mouse is seen by the image sensor 104. The particlesurface is largely represented by pixels 201, 204, 205, etc. Theparticle edge is largely represented by pixel 202. It is to be notedthat the difference in pixel values (i.e. the gray level) between pixel201 and pixel 205 is usually smaller than that of pixels 202 and 203.Thus, for the best result of pattern recognition, it is desired torecognize the boundary, sharp tip, etc. by optical means. In order toreach this goal, a previous effort resorted to lighting techniques tomake the object (i.e. surface textures, particles) stand out more fromthe background. The theory is the following: In materials science, theroughness of a surface is often denoted by an Ra value (specifically,roughness Ra is quantified by the arithmetic average of the verticaldeviations (i.e. yi) of a real surface from its ideal form, which isexpressed mathematically as

$R_{a} = \left. {\frac{1}{n}\sum\limits_{1}^{n}}\; \middle| y_{i} \middle| . \right.$Surface roughness has fundamental influence on microscopic images inthat it forms microscopic shadows for the tiny objects that stand“extruded” out of the targeted surface plane. When there are distinctiveshadows in the image, it is easier for the image sensor to see theobjects.

FIG. 3A shows an image taken by a conventional optical mouse that isilluminated by a tilted light (e.g. approximately 45 degree to thetargeted object surface). The shadow of the particle is represented asPixel P1, P2, and P3. Pixel P4 represents the top surface of the object.Note that the contrast between P1, the shadow, and P5, the pixelrepresenting the background, is usually larger than that of top surfaceof the object, P4. In fact, to the best performance of these types ofoptical mice, P4 is preferred to have as little variation as possible.In order to reach the goal of having the highest contrast for shadow andthe lowest one for the object top surface. A second light source can bea diffused light beam that shines on the pixel plane P4. Since thislight beam is preferred to be a diffused one; it is intended to form a“flat” image of the object surface and not used for pattern recognitionand background; therefore, the variation in pixel values in therespective areas is very little.

As the above cases illustrate, today's optical mouse uses monochromaticlight as a conventional mean to illuminate object, and this techniquehas lasted for decades. As a consequence seeing a color image has notbeen a requirement for the conventional optical mouse. Instead, it isthe capability of seeing the shadows or textures in the image that isthe most important to the performance of the conventional optical mouse.

U.S. Patent Application No. 2009/0160772 A1 (DePue et al.) is directedto an optical mouse with diffused optics, wherein an optical diffuser isconfigured to diffuse light from the light source that illuminates thetracking surface. Despite that this works well in ordinary situations inthe extreme occasions when the surface of the targeted object is verysmooth (i.e. Ra is too low), the approach fails in that there just areno or too few shadows available for the optical mouse to see at all.

There are some special designs to drastically increase the brightness ofthe particles or textures on object surface by enhancing the lightintensity by laser light source. There are other approaches that usedark field image to make the background looked dimmed, and there areapproaches that use a special angle of incident light to shine on theobject surface in hopes that the PSNR is enhanced (PSNR stands for thepeak signal to noise ratio). This is not only associated with thesurface roughness Ra, but also the spectral response of the objectsurface. In conventional optical mouse device, the ratio of theintensity of the light beam coming from the targeted objects to that ofthe background light can be interpreted as the signal to noise ratio ofthe picture. Alternatively, the mouse industry use a similar meaningfactor called the SQUAL, which denotes the surface quality. SQUAL countsthe number of features identified by an image frame. If the signal levelis increased caused by increasing the light intensity, or the backgroundsignal is suppressed by some diffusion means to a flat level with lessundulation then the PSNR is increased. So, many of the conventionaloptical mice endeavor to increase the PSNR by, for example, adjustingthe tilting angle of incident light timely, or using multiple lightbeams, etc. U.S. Pat. No. 7,439,954 B2 (Theytaz et al.) is directed to amulti-light source computer pointing device. U.S. Pat. No. 6,392,632(Lee) is directed toward an optical cursor controlling device comprisingan integrated camera. U.S. Patent Application No. 2005/0024336 (Xie etal.) is directed to the use of a Lambertian surface in a computer mouseto form a spectral reflection, which provides an enhanced contrast tothe image for purposes of navigation. U.S. Patent Application No.2005/0024623 (Xie et al.) is directed to the use of a Lambertian surfaceand a narrow bandwidth of light in a computer mouse to form a narrowbandwidth spectral reflection. U.S. Pat. No. 5,825,945 (Stolis et al.)is directed to a check imaging device using a Lambertian surface toproject a highly uniform and diffused beam to an imaging site.

FIG. 3B demonstrates the drawback in the conventional art, when anoptical computer mouse views the contour of an object on a desktopsurface. When the targeted desktop surface has a very low surfaceroughness value or is made of a glossy material, the number of pixelsrepresenting the shadows or a unique point is rarified (i.e. P6A andP6B); wherein the total number has been decreased from three to twopixels comparing FIG. 3B to FIG. 3A. Also note that the contrast of thegray level among P6A and P6A (i.e. the shadow). P7 (the backgroundscene), and P8 (the object body) has been decreased drastically. Theseeffects often lead to the erroneous result of motion detection where thegray level denotes the energy density that a pixel receives from lightimpingement which can be monochromatic or multi-colored, depending onthe sensitivity of the respective pixel.

SUMMARY

It is an objective of the present disclosure to provide a high speed,high resolution, sensitive, and low power optical mouse comprising a setof multiple color lighting sources;

It is further an objective of the present disclosure to adjust the lightsources of the optical mouse in accordance with colorimetry, wherein adigital image is dissected into several subimages in which each subimageis constituted by elements of one of the primary colors.

It is also an objective of the present disclosure to use at least threedifferent colored lights wherein each of the colored lights is a primarycolor.

It is also further an object of the present disclosure to use anoptoelectronic sensor, for instance an image sensor in which a CFA(color filter array) formed over the individual pixel diodes matches thecolor of the light sources of the optical mouse.

It is still further an objective of the present disclosure to be able tocontrol the intensity of each colored light independent of the otherlights.

In the present disclosure, multiple color lights are used, wherein thespectral energy density of the lights have been recorded, oracknowledged by a program embedded in the light control circuitry of theoptical mouse. When the sources of light shine on an object surface, thecontrol circuitry adjusts the intensity of each of the respectivesources independent of each other, as well as the characteristic of thelight sources (e.g. power density).

The most unique and important part of the disclosed optical mouse isthat the control algorithm adopts colorimetry; the term colorimetry asis disclosed by this disclosure has at least the following meaning.

-   -   (1) Spectrally different stimuli combine, mathematically or        physically, to yield new stimuli that can be used by different        devices (e.g. LED, color filter, computer program, optical        mouse, electronic displaying device, etc.).    -   (2) Three primary stimuli form the basis of stimuli in color        space, which can be red, blue, and green; other stimuli may be        YMC, YUV, etc.    -   (3) A complete set of all allowable stimuli forms a color gamut.    -   (4) Colorimetry has to do with the accurate expression of color.        It is not a conceptual term. Data generated per colorimetry        teaches the engineer how to manipulate the colors for different        color rendering devices.    -   (5) The transformation process from one set of primary stimuli        to another is done by the following:

$\begin{matrix}{\begin{bmatrix}D \\E \\F\end{bmatrix} = {\begin{bmatrix}D_{{A = 1},{B = 0},{C = 0}} & D_{{A = 0},{B = 1},{C = 0}} & D_{{A = 0},{B = 0},{C = 1}} \\E_{{A = 1},{B = 0},{C = 0}} & E_{{A = 0},{B = 1},{C = 0}} & E_{{A = 0},{B = 0},{C = 1}} \\F_{{A = 1},{B = 0},{C = 0}} & F_{{A = 0},{B = 1},{C = 0}} & F_{{A = 0},{B = 0},{C = 1}}\end{bmatrix} \times \begin{bmatrix}A \\B \\C\end{bmatrix}}} & {{EQ}\mspace{14mu}(1)}\end{matrix}$For example, the typical primary stimuli

$\begin{bmatrix}A \\B \\C\end{bmatrix}\quad$used by today's color CMOS image sensor is R (red), G (green), and B(blue). By transformation, RGB can be transformed in to YUV, Lab, etc.The primary stimuli system used for H.264 video streaming file, adoptedby an iPhone (manufactured by Apple, Inc.) and better for motiondetection use, is YUV. Thus, it is a common process for today's colorimage sensors that, prior to doing motion detection, a transformationfrom the stimuli as received into a different one takes place.Associated technology in this field is often referred as colormanagement. As can be noticed easily, color management is done based oncolorimetry, which has to do with the accurate expressing of colors.

Based on colorimetry, the controlling circuitry of the presentdisclosure knows that when one primary color in the captured imagechanges its power intensity, the spectral performance of the otherprimary colors as seen by the image sensor do not change. Thus, thedisclosed optical mouse only has to adjust the power density of thelight source associated with one primary color at one time, which willnot change the result of the other primary colors. Therefore, the methodis robust and comprehensive in enhancing the resolution and sensitivityof the optical mouse on all kinds of surfaces.

Transforming one stimuli system to another is practiced by many of thedigital cameras where certain stimuli system can be more sensitive thanothers in specific applications (e.g. video format YUV 4:2:2 is moresensitive to brightness, video format YMC is more addressable to yellowcolor, etc). It should be noted that under the optical mouse, the smallcavity being faced by the image sensor is literally a dark room, variouskinds of optical effects such as light absorption, transmission,Lambertian diffusion, florescence, scintillating, etc. may happen. Theseeffects may not be easily observed in the daylight environment, but whenthat dark room is placed on a carpet, granite, wood, nylon, etc, theymay become the source of artifacts not addressable by the conventionaloptical mouse.

In optical science, three primary colors suffice to represent the wholecolor gamut. Using more primary colors than three is acceptable, but forthe clarity of explanation, the present disclosure uses only threeprimary colors in the embodiments. This means that the presentdisclosure is able to adjust the apparent color of the object in acontinuous mode. Mathematically, there is always a middle gray valuebetween two gray level values, thus the color tone can be adjustedcontinuously. The second advantage is that the apparent color of theobject can be adjusted by the illuminating light sources over a widerange (i.e. the entire color gamut), and this allows the disclosedoptical mouse to work almost on any kind of surface, so long as thatsurface can reflect the incident light by a satisfactory amount.

The process associated with colored light control is very simple, rapid,power economical, and efficient in directing motion, since it does nothave to worry about picture quality. What the control circuitry caresfor is not PSNR (peak signal to noise ratio), but instead, the dynamicrange of a subimage and the effectiveness of pattern recognition. Asubimage is a view of an object only populated with one of the primecolors used in the computer mouse of the present disclosure. In the caseof three prime colors there will be three subimages, one for each color.

An enhanced dynamic range of the subimages allows for the disclosedoptical mouse to find patterns at high sensitivity and high resolution,and in turn makes the disclosed device outpace the conventionalmonochrome optical mouse by higher sensitivity and resolution. Likewise,when an illumination system is set in a way that is favorable to patternrecognition (e.g. less smearing effect on the contour), the overallperformance of the motion detection device is enhanced.

The most prominent feature of the disclosed optical mouse lies in thecollaboration between its image sensor and the controlling circuitry,which uses the data provided by the image sensor to adjust the lightingcondition of the multiple light sources, where the disclosed colormanagement methods (i.e. the embodiments of the present disclosure) arederived based on colorimetry, which are intended to address thecomplicated problems that arise from the fundamental physics (e.g.diffusion, absorption, florescent, etc.) without knowing, for example,how a granite desktop surface interacts with an LED light. With this keyfeature, the disclosed optical mouse is able to capture the delicatechange on the object surface (i.e. color) without resorting to surfaceroughness or edges of the objects. The present disclosure outpaces thestate of the art in which there has not created any effective mean toadjust the illuminating condition judiciously, and rapidly. The humaneye can perceive millions of colors, and the objects surrounding us areoften rendering a mixture of multiple colors (e.g. the color of adesktop plate made of poplar is a mixture of red and yellow colors).Without a clear rule as how the multi color lights are to be adjusted,those approaches using multiple color may find little application in thereal market. For example, the color of a desktop surface can be apoplar, mahogany, etc., represent different combination of red andyellow color, and in this case when illuminated by the daylight, or aD65 CIE standard light source, the poplar surface would have littlecolor ingredient in blue. Whenever facing such a situation, existingtechnology that claims to use multi-color light in an optical mouse failin that they cannot point in any direction to enhance the resolution andsensitivity of the optical mouse by the multiple light sources.

BRIEF DESCRIPTION OF THE DRAWINGS

This disclosure will be described with reference to the accompanyingdrawings, wherein:

FIG. 1 is a diagram of a conventional optical computer mouse using onelight source;

FIG. 2 depicts an exemplary pixel map of an object seen by theconventional the optical computer mouse;

FIG. 3A depicts an exemplary pixel map of an object seen by theconventional optical computer;

FIG. 3B depicts an exemplary pixel map of an object surface with lowsurface roughness and shadows with reduced grey levels;

FIG. 4A is a diagram of the optical computer mouse (motion detectiondevice) of the present disclosure using multiple light sources;

FIG. 4B is the circuit schematic of the optical computer device (motiondetection device) of the present disclosure, which is activated bymultiple light sources;

FIG. 5 is an exemplary pixel map of an object activated by multiplelight sources and seen by the optical computer mouse (motion detectiondevice) of the present disclosure;

FIGS. 6 A, B, C, and D depict an exemplary process of dissecting a mainpixel image into multiple subimages;

FIGS. 7A, B, C, and D show shadowing resulting from the multiple lightsources of the preferred embodiment of the present disclosure;

FIG. 8 depicts an exemplary array of photodiodes whose color sensitivityhas been assigned specifically to red, blue, and green light;

FIGS. 9A, B, C, and D depict an exemplary case of an object havingstrong spectral response to red and blue light but no response to greenlight;

FIG. 10 shows the spatial arrangement of the lens and color imagesensor. and depicts a unique phenomenon of multiple shadows;

FIG. 11 shows the pixel images of the shadows formed by different colorlight sources as they appear in different subimages:

FIG. 12 shows how the relative position between a targeted object andthe image sensor changes over two different locations over two differenttimes;

FIG. 13 shows resultant images taken by the color image sensor at twodifferent times:

FIG. 14 depicts the geographical relationship among multiple lightsources, a color image sensor and the targeted object on a surfaceplane;

FIG. 15 depicts an exemplary exhibit of how the gray level of each colorsensitive photodiode varies when the optical mouse of the presentdisclosure moves over a multi-colored desktop surface;

FIG. 16 depicts an envisioned image frame of a wood surface with annularrings captured at time t3;

FIG. 17 depicts an envisioned image frame of a wood surface with annularrings captured at time t4;

FIG. 18 shows the method and controlling circuitry and the illuminationcondition of multiple color light sources of the present disclosure:

FIG. 19 shows the switching condition for the light sources of thepresent disclosure;

FIG. 20 shows a circuit diagram of a photodiode of a pixel that iscapable of controlling dynamic range and resolution on individual pixelbase; and

FIG. 21 shows the color gamut used by the disclosed optical mouse.

DETAILED DESCRIPTION

In FIG. 4A is shown an optical electronic device, for example an opticalmouse, 402 of the present disclosure, which can be connected by aninformation carrier 406 to a computer either by wireless means orphysical means using a USB cable or equivalent. The optical mousefurther comprises a cavity 408 containing a plurality of colored lightsources 403, 404, and 405 and an optoelectronic sensor (an image sensor)407. It should be noted that there should be three light sources as theoptimal condition, but more or less light sources can be used (e.g. awhite light LED can emit red, green, and blue light beams from a singlediode covered by several kinds of dyes). A first light source 403, asecond light source 404, and a third light source 405 impinge light ontoa targeted desktop surface 409, wherein each light is of a differentcolor. When the light reflected from the targeted desktop surfacearrives at the optoelectronic sensor, for example a color image sensor407, it forms an exemplary image as shown in FIG. 5, which shows a fullcolor picture constituted by three kinds of pixels, red, blue, andgreen.

In FIG. 4B is shown the circuit diagram of the disclosed optical mouse.A digital image sensor 407 captures an image at a predetermined framerate. The captured image is dissected into several subimages, e.g. a redsubimage, a blue subimage, and a green subimage. The subimages can beeither sent to the memory 414 for storage, or to the image processor 413for motion detection use. The pattern recognition process is done in theimage processor 413 at high speed. The I/O controller communicates withthe computer to which the optical mouse is attached through the I/Ocontroller 415. An example of separating the captured image, FIG. 6A,into subimages is shown in FIGS. 6B, 6C and 6D, each of which representsa scene captured by a special color scheme, the color red FIG. 6B, thecolor green FIG. 6D, or the color blue FIG. 6C. When combined together,the “patched” pixel array represents the complete image FIG. 6A of theobject, but for motion detection use, this combining process usually isnot necessary. When performing motion detection tasks, image processor413 compares a series of subimages in one color scheme. The imageprocessor 413 does not compare one subimage FIG. 6B to another that hasa different color scheme FIG. 6C, unless there is any special purposenot addressed by the present disclosure.

The disclosed optical cursor maneuvering device may resemble the formfactors and geometrical characteristics of the conventional opticalmouse (i.e. case, camera, cavity, and light source, etc), but as FIG. 4Ashows, there is a cavity 408, that contains a plurality of colored lightsources 403, 404, and 405 formed from, for example, LED (light emittingdiode) devices each of which emits a light with a wavelength spandifferent from that of the others LED devices. Inside the cavity 408 ismounted an image sensor 407 that is able to detect color images, thecavity 408 can be formed into different shapes, e.g. square,rectangular, circular, oval, or an irregular one. The plurality of lightsources are preferred to be positioned at a maximum separation in thecavity 408, and the image sensor 407 is preferred to be positioned onthe base of the cavity 408 amongst the plurality of colored lights 403,404, and 405 in such a way as to allow the image sensor 407 to capturethe image of the object illuminated by all light sources as evenly aspossible.

It should be noted that although the plurality of light sources ispreferably LED devices, there are other light sources (e.g. laserdevice, etc) that can be used. It should also to be noted that the imagesensor 407 is preferably, but not limited to, a CMOS imaging sensorcapable of capturing color images and dissecting the images capturedinto multiple subimages, each of which is characterized by a primarycolor, e.g. red, blue, or green. Note also that the primary color is notlimited to red, blue, and green. For example, the primary color set canbe yellow, magenta, and cyan, or other colors as well. The lightingcondition of the light sources 403, 404, 405 is controlled by a lightcontrol circuit 412, which adjusts the output power of the light sourcesbased on the feedback from the image processor 413 in a timely fashion.

FIG. 8 shows how the photodiodes of image sensor are designated todifferent primary colors. This is often referred by people familiar withthe art as the Bayer structure. There are other ways of designating thecolor sensitivity of each photodiode, and each may have its own purpose(e.g. Honeycomb type color filter array, etc.). For the clarity ofexplanation in our disclosure, the Bayer structure is used. Other waysof adjusting the color sensitivity of photodiode is applicable to thisdisclosure since the same method to adjust the lighting condition can beused. In concurrent digital image sensor industry, the task of assigningthe photodiode to different color is mostly done by placing a colorfilter array (CFA) over the image sensor. There are other alternativeways of designating the color sensing capability to the photodiodes(e.g. using ion implantation or even band gap engineering to adjust theinternal quantum efficiency of the photodiode, etc.), but using CFA isby far the most popular mean in today's digital image sensor industry.The typical method of assigning the primary colors (i.e. Red, Green, andBlue) to the pixel frame is referred as the Bayer structure. Accordingto Bayer structure, within any area composing of four photodiodes as arectangle (or square), two photodiodes are to be assigned to greenlight, one photodiode to the red light, and one photodiode to the bluelight. There are many alternative ways to arrange the photodiodes.

FIG. 9A shows an exemplary situation that the present disclosed opticalmouse sees an object in purple color. In this case, the red subimage(FIG. 9B) and blue subimage (FIG. 9C) have higher signal values (i.e.the grey level of the respective photodiode) than the green subimage(FIG. 9D) does. Referring back to FIG. 4B, for saving power, the lightcontrol circuit 412 of the optical mouse may reduce or even turn off thegreen light source 405 for the object in purple color. The imageprocessor 413 will acknowledge this, and not pick the green subimage(FIG. 9D) for motion detection use. It should be noted that FIG. 9A onlypresents the pixel array. The intensity of color light impinging on theeach pixel may vary, and they are represented by the grey level of R1through R6, B1 through B4, as is shown in FIGS. 9B and 9C. For bestpattern recognition result, it is desired to have the largest differenceamong R1 and R6, or among B1 and B4. The ability to pick out an imagethat is only associated with a specific spectral performance of thetargeted surface (e.g. reflectance, florescent, etc.) is an advantageover existing technology since existing technology only usesmonochromatic light.

When performing pattern recognition task, the optical mouse of thepresent disclosure may compare the red subimages or the blue subimages,or both but is usually not necessary. While the optical mouse is movingon the desktop, the gray level of the red subimage (FIG. 9B) and that ofthe blue subimage (FIG. 9C) may change due to the spectral response suchas the reflectivity to one color of light of the desktop surface, whichvaries from place to place. If the gray level of the red image goes toolow, then the controlling circuitry will increase the intensity of thered light, and, according the physics of light, such an act as onlyincreasing the intensity of red light will not lead to any change, orjust a negligible amount of change on the blue subimages. Thus, whilethe disclosed optical mouse is moving, the controller circuit canquickly adjust the power sent to the red light source. As for the bluesubimage, it can be left unused, or be investigated as a backupreference, which at certain times is required by the image processor.The same process can be done on the blue light source leaving the redlight source as the backup. Therefore, the disclosed optical mouse maycome to an optimized setting for the output power of the respectivelight sources based on the spectral performance (e.g. reflectance,florescent, etc.) of the desktop surface, and this optimized setting canbe changed at high speed. As another example, whenever facing a desktopsurface that has a “piebald” or an assorted structure characteristic,the disclosed optical mouse can accommodate itself to a few presetvalues, depending on where the optical mouse is placed, and theknowledge of how to fine tune the illuminating condition in an efficientand rapid manner.

The following explains how the light source control circuit 412recognizes/adjusts the intensity of light, and explains how the signalsin primary colors are measured. In optical physics, Equations EQ. 2through EQ. 5 can be used to calculate the spectral power density of thelight detected by an image sensor, wherein:

$\begin{matrix}{X = {k_{\lambda}{\Sigma S}_{\lambda}R_{\lambda}\overset{\_}{x_{\lambda}}{\Delta\lambda}}} & ({EQ2}) \\{Y = {k_{\lambda}{\Sigma S}_{\lambda}R_{\lambda}\overset{\_}{y_{\lambda}}{\Delta\lambda}}} & ({EQ3}) \\{Z = {k_{\lambda}{\Sigma S}_{\lambda}R_{\lambda}\overset{\_}{z_{\lambda}}{\Delta\lambda}}} & ({EQ4}) \\{k = \frac{100}{\underset{\lambda}{\Sigma}S_{\lambda}\overset{\_}{y_{\lambda}}{\Delta\lambda}}} & ({EQ5})\end{matrix}$

X, Y, and Z are color index values (the stimuli) representing the energydensity of photons in the resultant spectrum received by the imagesensor:

S is the intensity of light emitted by the light sources;

R is the reflectance of light by the object being viewed by the imagesensor;

□ is the wavelength of the color light; and

x, y, z are the sensitivities of the image sensor at a specificwavelength □□

In a CMOS image sensor, x, y, z are often related to the color filterdesign (e.g. Bayer color filter). As EQ 2 through 5 shows, the colorindex X, Y, and Z, are irrelevant to each other when they are integratedover different ranges of λ. Putting it succinctly, when the presentdisclosed color optical mouse uses multiple lights with differentwavelength spans, the resultant value of X, Y, and Z values areindependent to each other. The benefit of such an independence isimportant since if one increases the intensity of one light source, onlyone parameter among X, Y, and Z will change. Thus, the disclosed opticalmouse is able to dissect an image into three subimages per the primarycolor system used i.e., red, green, and blue image, or whatever primarycolors that are chosen to be used.

EQ.2 through EQ.5 also denotes that based on the same physics, there canbe other ways to adjust the sensitivity of the present disclosed devicethan adjusting the power output of respective light sources. Forexample, in the optoelectronic industry, the LED devices that emit lightin different color may use essentially the same kind of diode coveredwith different types of dyes to get different apparent colors. Thus,adjustment of the relative intensity of the respective light beams canbe accomplished by manipulating the chemical ingredients within the dye.Such a method is in compliance with the general rule taught by EQ 2through EQ5. As another example, the color sensitivity of the respectivepixels in a typical CMOS image sensor is influenced by the genericnature of the color filter materials (e.g. CFA), which are essentiallychemical materials that only allow the light beams in specificwavelengths to pass through. By adjusting the generic nature of therespective color filter material, the sensitivity of the presentlydisclosed device can also be adjusted. The present disclosure teaches ageneral method to adjust the sensitivity of a cursor maneuvering deviceusing light beams in different colors or wavelengths. There areadditional methods that can be derived based on the same design ruleprovided by the present disclosure.

There are other ways to adjust the sensitivity of a photodiode. Forexample, in conventional art there is a technique called variablesensitivity photo detector (VSPD). This technique combines two pixelarrays whose data are stored in the inverted manner based on two verysimilar image frames like two pictures taken by an optical mouse withina small time interval, like ns (nanoseconds), and while the mouse ismoving; combining two tone-inverted images that would cancel out most ofthe chunk body of the object and only leave the edges of the objectobservable in the resultant image. This technique has been usedpreviously to perform edge detection tasks. What VSPD uses is amonochrome image, which does not exploit the rich information that isprovided by means of colored light. Nevertheless, various means providedby previous technology for enhancing the capability of an optical mousein edge detections, or pattern recognitions, such as VSPD, mask filters,etc., can be adopted by the present disclosure as an affiliating methodto further enhance the fundamental performance in performing highsensitivity and high resolution motion detection task using multiplecolor lights.

Most color filters uses Bayer color filter array. A Bayer CFA iscomposed of a large number of micron-scaled color filters placed ingrids in front of the photodiodes in mosaic manner. Each unit grid isassigned as red, blue, and green color, but not limited by these colors.There are other methods of assignment such as magenta, cyan, and yellow.The present disclosure document uses RGB as an example. As explainedprior, in a Bayer CFA, each unit cell is positioned in the grid andsurrounded by four other cells, but the frequency distribution of eachof the colored lights should have the least amount of overlap (so thereis a minimum of color mixing as possible). Attributed to the genericcharacteristics of the color filter material, a photodiode designated tored color is only sensitive to red colored light, and it is effectivelynot sensitive to green and blue light. A CMOS image sensor can easilyreach a capacity of mega pixels. When an optical mouse adopts a highcapacity color image sensor and a CFA, its resolution and sensitivity isgreatly enhanced as compared to that of conventional optical mouse. Thecollaboration between multiple colored light sources and the color imagesensor, as shown in EQ. 2 through EQ. 5, offers an advantage that havenot been attained previously.

FIG. 21 shows an exemplary color gamut used by the disclosed opticalmouse. The color gamut is formed by a spectral energy locus of wavelengths of light ranging from 420 nm (nanometers) to 520 nm to 700 nm. Aline between 420 nm and 700 nm forms a purple line boundary 2105, andthe colors blue 2106, green 2104 and red 2103 are located in the gamutregion near 420 nm, 520 nm and 700 nm, respectively. On the color gamutthere are shown three triangular shaped color regions, each representinga different color models, NTSC (National Television System Committee)2109, RGB (Red, Green, Blue) 2108 and CMYK (Cyan, Magenta, Yellow andKey (black)) 2107. Within each the color gamut is a point of equalenergy 2101 that lies within the triangular shape color regions of thethree different color model regions.

According to FIG. 21, the three primary colors, red 2103, green 2104 andblue 2106, are sufficient to represent any point in the color gamut.This means that the disclosed optical mouse is able to adjust theapparent color of the object to any color tone it desires in acontinuous mode, and where continuous mode means there is always amiddle pixel value between two pixel values. In practice, the disclosedoptical mouse would choose a color tone setting, red, blue, and green,as RBG 2108 shows for a specific subimage such that the subimage givesthe widest dynamic range or the highest contrast between object andbackground. For example, for a poplar surface, the disclosed opticalmouse may try to increase the intensity of red light or yellow light, orboth, so that the dynamic range of the red subimage can be enhanced. Theblue subimage of the disclosed optical mouse may suspend the blue imageprocessing work since the reflectance of the poplar surface to bluelight is low. This actually makes the combined image frame of the red,green and blue subimages unbalanced to white color. A white colorbalanced illumination is produced when red, green and blue light is inequal weight, as point 2101 of FIG. 21 shows. However, such anunbalanced image provides unprecedented advantages for motion detectionuse.

In the present disclosure, the unbalanced illumination system can beadjusted in a timely fashion, so that the disclosed optical mouse alsoprovides another advantage for its high adaptability to environment.This is mainly attributed to the fact that the apparent color of thetargeted object can be adjusted by the light sources in a wide range(i.e. theoretically, the entire color gamut). The wide range of apparentcolor tone allows the disclosed optical mouse to work on any kind ofsurface, so long as that surface can reflect the incident light by asufficient amount.

More specifically. FIG. 21 is a general description of the color gamutbased on CIE1931, other color gamut may have a similar effect asdescribed by the present disclosure. In FIG. 21, the horizontal axis isx, the vertical axis is y. The relationship among x, y, and z (z is notshown in FIG. 21) is the following.

$\begin{matrix}{x = \frac{X}{X + Y + Z}} & ({EQ6}) \\{y = \frac{Y}{X + Y + Z}} & ({EQ7}) \\{z = {1 - x - y}} & ({EQ8})\end{matrix}$

Where: X, Y, and Z are the tri-stimulus values of a color represented bythe amounts (i.e. intensities) of three primary colors (e.g. R, G, B).

It should be noted that although the present disclosure is directed tocursor maneuvering, there are other utilities and functions (e.g. motiondetection, program activation, etc.) that can be derived from thedisclosure herein described that can be adopted by the electronicdevices such as cell phones, stationary electronic devices such asdesktop computers, etc.

Embodiment 1

FIGS. 7A, B, C and D shows a preferred embodiment of the presentdisclosure. As FIG. 7A shows, the bottom view of an disclosed opticalmouse comprises a cavity 718, inside the cavity 718 lies a color imagesensor 717 and three light sources, a first light source 714, a secondlight source 715, and a third light source 716. The mouse buttons 719can be placed anywhere on the mouse case 720 that is convenient to theoperator.

FIG. 7A shows that at least three color light sources are preferred tobe used, which are in different primary colors. In a specialarrangement, FIG. 7C shows a first light source 714, which emits a redlight, FIG. 7D shows a second light source 715, which emits a bluelight, and FIG. 7B shows a third light source 716, which emits greenlight. The light beam 724 of each of the light sources 714, 715 and 716cast a shadow 704, 705 and 706 of a surface variation 708 onto thedesktop surface 707. The shadows formed on the surface of the desktop 70are from the colored lights shining at, or across the surface variation708 comprising an imperfection, defect or a surface particle on thesurface of the desktop

FIG. 10 shows the spatial arrangement of the lens 1022 and color imagesensor 1023, which corresponds to item 717 in FIG. 7. In this preferredembodiment, the color image sensor 1023 is positioned at a verticaldistance of a few mm from the targeted desktop surface 1007. Thedistance 1008 from the targeted object plane surface 1007 to the lens1022 is typically a few mm to tens of mm. The three light sources 714,715, and 716, shown in FIG. 7A, are also positioned at a distance of fewmm away from the targeted desktop surface 1007 (707 in FIGS. 7B, 7C and7D). Referring back to FIG. 7A, the three light sources 714, 715 and 716are preferred to be positioned close to the perimeter of the cavity 718,so that an incident angle between the light beam 724 and the targeteddesktop surface 707 of approximately 45 degrees is formed. The threelight sources enclose an area having about the same area as that of thecavity 718, and the color image sensor 717 is positioned close to thegeometrical center the enclosed area. The benefit of this arrangement isthat shadows can easily be taken advantage of.

Almost all surfaces have obtrusive objects lying on them if a magnifyinglens is used to see the microscopic world. Obtrusive objects can be dustparticles, perturbations, variations or defects in a surface. Forexample, in FIGS. 7B, 7C and & D there is an object 708 lyingobtrusively on the targeted desktop surface 707 that produces shadows704, 705 and 706 that is attributed to the arrangement of the threelight sources 714, 715 and 716, respectively. In fact, shadows andbright spots are interplaying roles in the image. At certain angles, theobtruded object 708 reflects light beams in an intensity that issubstantially higher than that of the neighborhood. Together, theshadows and bright spots form an image in high dynamic range. Forsimplicity, the following discussion uses shadows to explain how thisembodiment works, and the same physics should apply to the bright spots(not shown in FIG. 7).

It should be noted that the spatial arrangement of three light sourcesof present disclosure allows for any object to firm at least one shadow,or one bright spot, on the targeted desktop surface as shown in FIG. 10,wherein an obtrusive object 1006 projects three bar shaped shadows 1003,1004, and 1005 on the desktop surface 1007, spaced at approximately 0degrees, 120 degrees, and 240 degrees, respectively, as a result of thelocation of the colored lights. Attributed to the spatial arrangement ofthe multiple light sources, an object on the targeted plane surface 1006will form multiple numbers of shadows 1003, 1004 and 1005, each of whichis associated with one light source but not the other light sources.This phenomenon occurs with all objects on the targeted desktop surface1007, which becomes more obvious when the surface roughness of thedesktop surface 1007 is increased.

As previously mentioned, the three color light sources are in differentcolors 714, 715, and 716 and the image sensor 717 (1023) is colorsensitive. Hence, what is captured by the color image sensor 1023 inFIG. 10 can be represented by FIG. 11; a full color image that can bedissected into three subimages, 1108, which is formed by red pixels,1109, which is formed by blue pixels, and 1110, which is formed by greenpixels. In each mono-color pixel plane, the object body 1006 shown inFIG. 10 is represented by a group of pixels which is labeled 1121, forexample. There are three shadows 1111, 1112, and 1113 positioned aroundthe perimeter of the pixels representing the targeted object 1121. Itshould be noted that in each subimage 1108, 1109 and 1110, there is onlyone shadow, and each shadow is denoted by a specific color determined bythe light source creating the shadow.

FIG. 12 shows the relative motion of the disclosed optical mouse over adesktop. A point of interest (e.g. an object that can be used by themouse for motion detection) is that object, which changes position onthe image of the image sensor 717 (1023) while the disclosed mousemoves. For example, at time t1, point X is positioned on the right ofthe color image sensor 1201; after mouse movement, point X is positionedto the left of the color image sensor 1201.

The image seen by the color image sensor 1201 is shown in FIG. 13. Attime t1, a full colored image 1308A is dissected into to three subimages1302 A, 1304A, and 1306A, and the color tone of the respective subimagesis, red, blue, and green. The pixels representing point X, the targetedobject, is denoted as item 1301A. When time slips to t2, the image takenat time t2 1308B shows, the pixels representing the targeted object1301B has been shifted to the right in the subimage 1302B. In thissituation, shadow 1303A and shadow 1303B serve the best source forpattern recognition because they form the highest image contrast withthe pixels representing the targeted object 1301A and 1301B. However,there are occasions that the shadows are not available. As subimage1304B shows, the shadow 1305A formerly seen in subimage 1304Adisappeared. In this case, the shadow may have moved out of the view ofimage sensor, but there can be other reasons such as surface morphologychange. Thus, relying on one monochromatic source to track motion alwaysfaces exceptional situations, as most computer mouse users haveexperienced. In the example shown in FIG. 13, subimages 1302B and 1306Bcan be used whenever subimage 1304B is not available for patternrecognition use, wherein the shadow 1307A in subimage 1306A is alsovisible at time t2 in subimage 13068. Most importantly, subimages 1302B,1304B, and 1306B do not interfere with each other in that their imagesare formed by different color light sources, and the respective pixelsare not sensitive to the colors that are not in their sensiblewavelength span.

As a reiteration of one the most important advantages acquired by thisembodiment, the disclosed device uses multiple light sources, each indifferent color, as shown in FIG. 14, which depicts the geographicalrelationship among multiple light sources 1401, 1402, and 1405, colorimage sensor 1406, and the targeted object surface plane 1403. Lightsources 1401, 1402, and 1405 shine light on the focal point of imagesensor 1404. The light from each light source is separated by angles □,□, and □ on the targeted object surface plane 1403, where each angle isapproximately □□□□. The angle of inclination of the light from the threelight sources is approximately □_(□)□□₂□□_(□)□□□□□□ Thus □□ the lightsources are preferred to be positioned at a lateral distance ofapproximately a few mm away from the focal point of image sensor 1404,and further it is preferred that these light sources are positioned asfar apart from one another as possible. However there are other criteriathat could change the value of the inclination of the light beam, theergonomic design of the optical mouse of the present disclosure.

Embodiment 2

The second embodiment deals with a situation deemed by the state of theart as the most challenging, a very smooth, or a glossy desktop surface.FIG. 15 shows an exemplary case where the disclosed optical mouse 402(detailed previously with respect to FIG. 4) is placed on a very flatsurface 1509, such as a wood board painted with a glossy coating such aslacquer. The glossy coating is highly transparent, very flat, and canreflect a lot of light to the image sensor 407, known as smearing, at acertain incident angle, which makes the image processing tasks of theconventional optical mouse difficult. The flat surface 1509 made oflacquer does not provide the surface roughness condition wanted byembodiment 1, but the spectral data of the wood surface acquired by thesubimages still can serve as sufficient data sources for motiondetection. FIG. 15 shows an example where a poplar wood board 1509 hasannular rings 1512 with dark stripes that are brown in color. The restof the area between the annular rings 1513 is yellowish in color, mixedwith some red ingredient therein. While the disclosed optical mouse ismoved along the wood surface, the intensity of the red light reflectedfrom targeted desktop surface 1509 to the color image sensor 407undulates accordingly. This phenomenon happens on both the red and greenpixels. Basic optical physics teaches that yellow light can be acquiredby mixing red light and green light. Therefore, while the intensity ofred color light undulates, the green color light does as well. Whatneeds to be noticed is that the blue color does not change throughoutthe entire mouse moving action as the resultant data shows in FIG. 15.While the mouse is moving on the targeted desktop surface 1509 and theintensity of three light sources 403, 404, and 405 are kept constant,the intensity of the red light as perceived by the image sensor 407varies from a maximum to a minimum value from time t1 through t5. Thesame situation happens on the green reflected light, which variesbetween a maximum and a minimum in the interval between t1 and t5. Theblue light, on the other hand, maintains at a constant level very closeto the minimum.

FIGS. 16 and 17 show a portion of the pixel array of the image sensor407 at time t3 and then at time t4 and the optical mouse of the presentdisclosure is moved along the surface of the wood 1509. At time t3, astripe of brown annular rings is formed on a first column of pixels 1601(R1-G5-R5-G13-R8-G20); and the yellowish structure between the brownannular rings is formed on a second column 1602 (G1-B1-G9-B5-G17-B9).The situation is reversed at time t4 shown in FIG. 17 where the firstcolumn 1702 (R1-G5-R5-G13-R8-G20) now is designated to the yellowishstructure of wood surface between the brown annular rings, and thesecond column 1701 (G1-B1-G9-B5-G17-B9) now is related to the stripe ofa brown annular ring. Thus, in a series of subimages of red and greencolors, it is seen that the objects are moving, which cannot be donewith the conventional art. In the conventional art, a monochromaticlight source and a monochromatic image sensor is used. Thus, in aconventional optical mouse, a lumped image sensor would detect thecombined intensity of red and green light, which would make the signallevel of the conventional art not undulate and in particular withindependence between colors. As a consequence, the task of patternrecognition is stampeded. This embodiment of the present disclosuredissects a captured color image into multiple mono-colored subimages. Bydoing so, the sensitivity and resolution of the disclosed optical mousesurpasses the capability of conventional art.

Embodiment 3

This embodiment deals with a method of controlling illumination withmultiple colored light sources with the necessary circuitry. By dimmingone light, or enhancing the other, the disclosed circuit not only savesthe energy, but also enhances the sensitivity and resolution for motiondetection use.

We again start with the illustration of basic physics. It is known thatR+G+B=W, where R is red light, G is green light, B is blue light and Wis white light. For the optical mouse of the present disclosure, seekingwhite balance among R, G, and B color is not necessary; however, it isimportant that at least one subimage of a primary color present aresponse that varies between a minimum and a maximum as the computermouse is moved over a surface. In FIG. 15 is shown the response to threeprimary colors, red, green and blue, where there is a response variationin the red and green primary colors detected by the image sensor 407.Because the color of the surface over which the optical mouse is placedis not of a color containing a blue content, the blue response detectedby the image sensor 407 is a constant and at or close to the minimumlevel for blue reflected light intensity. As previously discussed, thesurface over which the optical mouse is moved is a highly reflectivesurface created over a poplar wood board. The wood board has annularrings that are brown in color with a yellowish color in between. Thusthe image sensor sees variation in the red and green light intensitysince it is red and green light that produces yellow and blue is not acomponent of the colors of the poplar board.

FIG. 18 shows a light control circuit 1811 of the present disclosure.The light control circuit 1811 controls the intensity of each lightsource in accordance with an input from the digital signal processor1810, where the digital signal processor receives data from image sensor1816. An exemplary controlling method is shown in the light controlalgorithm 1815. Note that there are cases in the commercial market thatthe DSP 1810, the light control circuit 1811, the I/O control circuitry1812, and the image sensor 1816 are merged as a single chip. There arealso other applications that modify these functional blocks partly, butthe essential purpose is the same, adjusting lighting condition based oncolorimetry.

In FIG. 18, a table of data 1807 sampled from the color image sensor isshown. Each column represents a series of data monitored by the DSP(digital signal processor) 1810 for illumination control in a period ofimage capturing over the time period t1 through t5, as shown in FIG. 15.This is an exemplary table where the disclosed device faces a situationsimilar to that of FIG. 15. Thus, at t3, when the mouse sees the brownstripes of annular ring, the red pixels R1, R2, R5, and R6, get highsignal. At this time, green pixels G1, G5, G6 and G9, have therespective signal output as H, L, L and H where H=High, close toMaximum; L=Low, close to Minimum. When time slips to t4, the mouse hasmoved to a new position which is full of yellowish structure, the redpixels change their level to L, L, and L; whereas those of the greenones change to L, H, H and L, respectively. The blue pixels don't changetheir pixel signal levels between time t3 and t4 because there is littleor no blue content in the poplar wood that makes the table top and theblue levels are always close to Minimum. The light control circuit 1811can check the variation of the red, blue, and green signals in a timelymanner. To assure the result is useful to multiple images, the DSP 1810monitors the signal level of multiple red pixels instead of one redpixel, and it also monitors the blue pixels and green pixels in the samefashion. Therefore, table 1807 contains the variation of light intensityacquired from subimages at t1 through t5. In this embodiment, anexemplary algorithm 1815 (detailed below) is used by the mouse of thepresent disclosure to check the variation of signal levels for red,blue, and green light at a time set by the optical mouse.

Algorithm

$\left. {{if}\mspace{14mu}\left( {{\Delta\; t} > t_{lcp}} \right) \times \left\lbrack {\left( \left| {\Delta\;\overset{\_}{R}} \middle| {> 0} \right. \right) + \left( \left| {\Delta\overset{\_}{G}} \middle| {> 0} \right. \right) + \left( \left| {\Delta\;\overset{\_}{B}} \middle| {> 0} \right. \right)} \right\rbrack}\rightarrow{{color\_ change}{\_ enabled}} \right.$ Color_change_enabled${~~}{{If}\left( {{\Delta\;\overset{\_}{R}} = 0} \right) \times \left( {{\Delta\;\overset{\_}{B}} \neq 0} \right) \times \left( {{\Delta\;\overset{\_}{G}} \neq 0} \right)\mspace{14mu}{{then}\mspace{14mu}\left\lbrack {Red\_ light}\rightarrow{{Red\_ light}\mspace{14mu} —~—} \right\rbrack}}$${~~}{{{If}\left( {{\Delta\;\overset{\_}{B}} = 0} \right)} \times \left( {{\Delta\;\overset{\_}{R}} \neq 0} \right) \times \left( {{\Delta\;\overset{\_}{G}} \neq 0} \right)\mspace{14mu}{{then}\mspace{14mu}\left\lbrack {Blue\_ light}\rightarrow{{Blue\_ light}\mspace{14mu} —~—} \right\rbrack}}$${~~}{{{If}\left( {{\Delta\;\overset{\_}{G}} = 0} \right)} \times \left( {{\Delta\;\overset{\_}{B}} \neq 0} \right) \times \left( {{\Delta\;\overset{\_}{R}} \neq 0} \right)\mspace{14mu}{{then}\mspace{14mu}\left\lbrack {Green\_ light}\rightarrow{{Green\_ light}\mspace{14mu} —~—} \right\rbrack}}$ Return end_if

Where:

|ΔR|: The absolute value of the change of the intensity of red light.

|ΔG|: The absolute value of the change of the intensity of green light

|ΔB|: The absolute value of the change of the intensity of blue light

Δt: time elapsed since last color comparing process ended

t_(Icp): time interval between two color adjusting processes

color_change_enabled: entering a procedure called Color_change_enabled

x: Logic AND

+: Logic OR

( ): Process in parentheses takes higher priority (to get result) thanthe X and +do Red_light→Red_light—: Red light dimmed by one unit (e.g.mW); associated parameters (i.e. current, voltage, etc.) are stored asthe parameter Red_light Blue_light→Blue_light—: Blue light dimmed by oneunit (e.g. mW); associated parameters (i.e. current, voltage, etc.) arestored in the parameter Blue_light Green_light→Green_light—: Green lightdimmed by one unit (e.g. mW); associated parameters (i.e. current,voltage, etc.) are stored in the parameter Green_lightThe symbol “−−” can be changed to “++” in different algorithm toincrease the intensity of the light.

Before or after the evaluation for variation of signal levels, theillumination condition is fixed. For example, if at the moment ofevaluation, the disclosed mouse finds out that one of the signals doesnot change, for example the signals of blue pixels, or □B=0; where □Bdenotes the variation of blue signal, then the control circuit 1811 willdim the blue light gradually. This is done by the switches S1, S2, andS3, whose action in the time interval between t1 and t5 is depicted inFIG. 19. As FIG. 19 shows, the switch S1 for blue light, graduallycloses, by moving from H to L in steps, when time slips from t3 to t4.But the other two switches do not change their respective signal values.This algorithm is very sure that when it changes the illuminatingcondition for one color light source i.e. blue light, the result for theother two kinds of pixels, red and green, do not change. (or change by anegligible amount when the CFA of primary colors have overlappingspectra, which is attributed to its knowledge in basic optical physics.It is very important to know that, according to basic physics of light,R+G+B=W, and for achieving the best result for the optical mouse of thepresent disclosure, the white image is not desired because in suchsituation it only raises signal level for all pixels. Thus, thedisclosed optical mouse seeks a means to adjust at least one color oflight to its lowest possible signal level. The algorithm 1815 isdesigned for such a purpose. Of course, when the image is not as clear,the disclosed optical mouse may increase the intensity of light, justlike most other approaches do. What differentiates the presentdisclosure from the existing state of the art is that, determined by theknowledge in colorimetry, the intensity of all light sources is notincreased concurrently, which has not previously been disclosed.

Embodiment 4

This embodiment demonstrates a microelectronic device whosefunctionality resembles the essential concept of embodiment 3, whichadjusts the energy density impinged on the photodiodes for motiondetection use. What differentiates embodiment 4 from embodiment 3 isthat in addition to dimming one light, or enhancing the other,embodiment 4 adjusts by enhancing or reducing the sensitivity of therespective photodiodes (red, green, and blue) to enhance the dynamicrange.

Today most of the digital image sensors are designed in a way where allphotodiodes in a same pixel array are storing charges in a same timeinterval. thus, their dynamic range is not adjustable. This embodimentuses different time intervals that vary for each individual photodiode;it allows each pixel to adjust its dynamic range based on theillumination condition.

Referring to FIG. 20, this is an exemplary case where the photodiodesand image processing circuitry are placed in a single chip. There is aphotodiode 2001 that collects the charges generated by aphoto-generation effect. As the influx of light energy hν 2011 continuesimpinging photons on photodiode 2001, a steady rate of charge is takenout of photodiode 2001. The charge then passes through inverter 2003 togenerate an output signal with a specific voltage span. This outputsignal then reaches the delay circuitry 2004, and is temporarily storedas voltage V2 at the embedded capacitor C_(st). Delay circuitry 2004bears an timing interval, which can be designed by a counter, that willdisconnect capacitor C_(st) from photodiode 2001 when the delay timemeets a specific value. When the delay circuitry is opened,disconnected, the charges in the inverter 2003 will be directed to thesecond input node (denoted by the plus sign) of the differentialamplifier 2005, to establish a voltage value V1 thereon. Through thedifferential amplifier 2005, the two signals V1 and V2 are compared, andthe result is passed to the second differential amplifier 2007 as therepresentative of the illumination condition experienced by thephotodiode 2001. According to CFA allocation, the photodiode 2001 willbe receptive to only one primary color, red, green, or blue. After thecomparison result is stored in capacitor Cm 2008, the system may proceedwith the image capturing process throughout the entire photodiode array,a first image frame is formed. Repeating the same process as statedabove, the second image frame is formed. When the second image frame iscaptured, the charge stored in Cm is compared to the respective pixeldata in the first image frame, where the result denotes the variation ofillumination. When the variation of illumination condition exceeds apredetermined value, comparator 2009, when enabled 2012, sends out aflag signal 2010. The flag signal 2010 will be referred by the disclosedmouse system as a sign of motion of the targeted object, and it willalso be used by light control circuitry to adjust the lightingcondition. In the mean time, the saturation detection circuitry 2006,when enabled 2012, monitors the output signal level of photodiode 2001.If the signal level of photodiode 2001 has not reached the saturationlevel, or the difference of illumination condition between consecutiveimage frames is still negligible, the reset switch 2002 will not turnon, which allows the charge collection process of photodiode 2001 tocontinue without interruption. By capturing images throughout anextended period of time as a result of no interruption caused by fixedtime interval signals, photodiode 2001 enhances its sensitivity, dynamicrange, and resolution.

As FIG. 21 shows, which is attributed to color physics, the discloseddevice is able to adjust the illumination condition of multiple lightsources and the sensitivity of photodiode concurrently. Specifically,this is accomplished by the work of the light control circuit work basedon signal V1, V2, (FIG. 20) and the flag signal 2010. It should be notedthat the image quality of the combined image frame of all primary colorsis not a critical concern to this embodiment. For motion detection use,it is sufficient to have a few pixels with high dynamic range,resolution, and sensitivity while the quality of image does notnecessarily please the perception of the human eye.

While the invention has been particularly shown and described withreference to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade without departing from the spirit and scope of the invention.

What is claimed is:
 1. A device for monitoring motion, comprising: a) anoptoelectronic sensor; b) a plurality of lights, configured forsimultaneous emission, wherein each of said plurality of lights isconfigured to emit a light beam of adjustable power output in awavelength, a band of wavelengths or a color different from that ofanother of said plurality of lights; and c) a lighting control circuitconfigured to receive output from said optoelectronic sensor and, as aresult thereof, independently adjusting each said light beam poweroutput; wherein said optoelectronic sensor is an array formed of aplurality of pixels, wherein each pixel of said plurality of pixels ismade sensitive to a different wavelength emitted by said plurality oflights and reflected by a target on a surface illuminated by saidplurality of lights, by placement over said plurality of pixels of afilter array configured to selectively pass only designated lightwavelengths to said pixels or by manipulating a doping profile of eachof said plurality of pixels or by engineering the band-gap structure ofeach of said plurality of pixels or by optimizing the array geometry forsensitivity to selected wavelengths; and wherein said device evaluates avariation in a signal derived from different colored or wavelength-bandsub-images produced simultaneously by said plurality of pixels, when apower output from at least one of said plurality of simultaneouslyemitted light beams is independently adjusted by said lighting controlcircuit; and wherein a relative movement between said device and saidsurface being simultaneously illuminated by said plurality of lights isdetermined by evaluation of variations of signals from said pixels,wherein said variations are produced by the combined effects of saidsurface and said target on said surface on said adjusted combinations ofindividual wavelengths, bands of wavelengths or colors of said lightthat are reflected from the surface and impinge on said pixels.
 2. Thedevice of claim 1, wherein said relative movement is used to produce acursor maneuvering device.
 3. The device of claim 2, wherein said cursormaneuvering device is an optical computer mouse.
 4. The device of claim1, wherein said optoelectronic sensor is an image sensor.
 5. The deviceof claim 1, wherein said movement is detected as variations of differentcolored sub-images of microscopic shadows formed from perturbations onthe spectral reflectance to light, surface morphology, or dust likeparticles on a surface over which said optical electronic device isbeing moved.
 6. The device of claim 5, wherein said variations ofmicroscopic shadows are determined by signals from the pixels of saidshadows and an area surrounding the shadows.
 7. The device of claim 1,wherein said movement is detected by variations in reflected light froma surface comprising color variation.
 8. The device of claim 1, whereinsaid pixels are adjusted for sensitivity to the plurality of lights toenhance a dynamic range of light being detected by said optoelectronicdevice.
 9. The device of claim 1, wherein said optoelectronic sensor ismatched to the wavelengths of the plurality of lights.
 10. The device ofclaim 1, wherein said plurality of pixels and said plurality of lightsare matched to an RBG (red, blue and green) color model.
 11. The deviceof claim 1, wherein said plurality of pixels is matched to a color modelof CYM (cyan, yellow and magenta), or any other cohesive color schemethat spans a color gamut.
 12. A motion detection device, comprising: a)an image detector configured to detect a variable wavelength opticalpower spectrum and to dissect said spectrum into a group of sub-images,each characterized by a single color or wavelength spectrum; b) aplurality of variable intensity light sources each having a spectrum ofpre-defined wavelengths or each being characterized by a single colorwherein each light source is configured to illuminate a targeted objectsimultaneously with the illumination of said targeted object by eachother of said light sources in said plurality; c) an image processingsystem coupled to said image detector within said device and configuredto determine motion of said targeted object by separately tracking eachof said group of sub-images characterized by a single color orwavelength spectrum emitted by or reflected from said simultaneouslyilluminated targeted object and detected by said image detector;wherein, when there is a movement of the device with respect to asurface on which said targeted object lies, wherein said surface issimultaneously illuminated by said plurality of light sources, then aseries of sub-images each characterized by a single color or wavelengthspectrum is captured during said movement, wherein said targeted objectis within each of said series of sub-images and is tracked by eachsub-image within said series of sub-images in which each said sub-imageis formed from a combination of the colors of light of variousintensities incident on and reflected by said targeted object from saidplurality of light sources at a spatial distance from said light sourcesand the optical effects produced by said surface and said targetedobject on the colors or wavelength spectrum of said incident light. 13.The device of claim 12, wherein each said single color, characterizingsaid group of sub-images or said plurality of variable intensity lightsources, is a primary color.
 14. The device of claim 12, wherein saidsingle colors, characterizing said group of sub-images or said pluralityof variable intensity light sources, are RGB (red, green and blue), orCYM (cyan, yellow and magenta), or any combination of color that forms acolor gamut.
 15. A method of motion detection, comprising: a) providingan optical electronic device comprising an optoelectronic sensor formedof a plurality of photo-detecting units, each sensitive to a differentcolor or wavelength spectrum of light, and a plurality of variableintensity light sources, each characterized by a spectrum of variouswavelengths or characterized by individual colors to which said sensoris sensitive, wherein said light sources are configured tosimultaneously illuminate an object on a surface over which said deviceis moving, wherein said optoelectronic device further includes a controlcircuit configured to analyze data provided by said optoelectronicsensor and to use said data to control said intensities of said lightsources with which said object is simultaneously illuminated; b)capturing, with said optoelectronic sensor, a plurality of sequentiallyseparate images of said object, wherein each said image is formedsimultaneously of sub-images, each characterized by sub-images ofindividual colors, created by reflections or re-emissions of saidsimultaneously illuminating light sources from said object duringrelative movement between said device and said object; and c)determining said relative movement of said object by using said controlcircuit to adjust intensities of said light sources and analyzingeffects of said adjustments on subsequent sub-images of said object. 16.The method of claim 15, wherein said device is a computer mouse.
 17. Themethod of claim 15, wherein said plurality of light sources each emitlight in a primary color.
 18. The method of claim 15, wherein saidoptoelectronic sensor is sensitive to primary colors.
 19. The method ofclaim 15, wherein intensity of at least one of the plurality of lightsources is adjusted by said control circuit.